There has been a surge in open source tools for developing reinforcement learning-based autonomous cyber defense. This talk will walk through, 1) why attendees should be interested in this emerging field, 2) how to use these tools, 3) how to extend them for tailored applications, 4) how to evaluate performance claims, and 5) what it will take to have fully autonomous AI defenses in the future.